The effectiveness of a cache memory system is related to the organization of the cache memory. Three of the variables involved in a cache memory design are the cache size, the line size and whether the cache is a split or a unified cache. The cache size refers to the total amount of storage available. The line size refers to the number of words sharing a tag in the cache. All words within a line are moved into and out of the cache together. In a split cache, instructions and data are cached separately. By contrast, in a unified cache, no distinction is made between instructions and data.
These variables affect cache performance in manners which are not always straightforward. For example, as the cache size increases, cache performance also increases. However, the relationship between cache performance and cache size depends also upon many other factors, such as the software environment, including system software, code size, main memory architecture and latency.
Generally, a larger line size improves cache performance because, due to locality of reference, the larger number of words prefetched may reduce future memory accesses. However, for a given cache size, a larger line size decreases the number of lines which can be resident in the cache, thereby increases the possibility of thrashing.
Finally, since instructions and data exhibit different patterns of locality of reference, cache performance is affected by whether the cache memory system is organized as a split cache or a unified cache. In some applications, e.g. where a relative small block of instructions operates on a large randomized data space, a split cache has a performance advantage due to the higher instruction hit rates; in other applications, a unified cache has a performance advantage.
Thus, for a given application, an optimal cache configuration can be derived only after extensive evaluations. Such evaluations are necessary to assist the computer system designer to determine which cache configuration best meets his or her design objectives. Traditionally, evaluations are performed using simulations in software. However, such simulations are often limited in accuracy, flexibility and speed. Because of a computer system's inherent complexity, accurately modelling the operation of a cache memory in software is difficult. In particular, many asynchronous events of a real computer system, such as direct memory access (DMA) and exception conditions, are difficult to model in software.